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ABSTRACT 

PILOT  ABILITY  TO  UNDERSTAND  SYNTHETIC  VOICE 
AND  RADIO'  VOICE  WHEN  RECEIVED  SIMULTANEOUSLY 


A  prior  study  of  the  use  of  a  voice  synthesis  system  and  its 
implementation  in  the  F-14  aircraft  was  completed  by  Grumman  Aerospace 
Corporation  in  1981  for  the  U.S.  Naval  Air  Development  Center.  The  goals  of 
that  study  were  to  determine  if  synthetic  voice  messages  would  reduce  the 
pilot's  workload,  increase  his  effectiveness  during  the  mission,  and  to 
establish  the  hardware  impact  on  the  aircraft.  Recommendations  for 
additional  studies  to  use  voice  in  the  F-14  and  other  future  naval  aircraft 
were  made  as  well  as  for  investigations  into  related  areas  such  as  crew 
perception  and  discrimination  of  a  computer  generated  voice. 

Some  pertinent  questions  arising  from  the  F-14  study  concerned:  a,  the 
distinctiveness  and  intelligibility  of  a  computer  generated  synthetic  voice 
in  the  aircraft  hearing  situation;  b,  whether  or  not  the  pilot  can  separate 
radio-transmitted  human  voice  messages  of  interest  from  simultaneously 
generated  synthetic  voice  messages;  and  c,  the  optimum  method  of  presenting 
such  messages  through  the  headset. 

An  exploratory  study  was  designed  to  investigate  these  questions.  The 
intent  was  to  examine  differing  methodologies  in  presenting  the  stimuli  used, 
and  to  ascertain  trends  in  the  subjects  performance  data. 

Results  tend  to  support  the  belief  that  synthetic  voice  messages 
provide  an  effective  means  of  presenting  alerting  messages  and  data  to  a 
pilot.  Radio  voice  does  not  interfere  appreciably  with  reception  of 
synthetic  voice. 

Use  of  a  dichotic  listening  method  may  enhance  perception  and  under¬ 
standing  of  radio  voice  messages  when  received  simultaneously  with  synthetic 
voice  messages. 

Future  studies  will  use  these  data  to  establish  definitive  design 
recommendations  for  incorporation  of  synthetic  voice  in  aircraft. 
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PILOT  ABILITY  TO  UNDERSTAND  SYNTHETIC  VOICE 
AND  RADIO  VOICE  WHEN  RECEIVED  SIMULTANEOUSLY 


I .  BACKGROUND 

A  prior  study  of  the  use  of  a  voice  synthesis  system  and  its 
implementation  in  the  F-14  aircraft  was  completed  by  Grumman  Aerospace 
Corporation  in  1981  for  the  U.S.  Naval  Air  Development  Center,  The  goals  of 
tha*.  study  were  to  determine  if  synthetic  voice  messages  would  reduce  the 
pilot's  workload,  increase  his  effectiveness  during  the  mission,  and  to 
establish  the  hardware  impact  on  the  aircraft.  Recommendations  for 
additional  studies  to  use  voice  in  the  F-14  and  other  future  naval  aircraft 
were  made  as  well  as  for  investigations  into  related  areas  such  as  crew 
perception  and  discrimination  of  a  computer  generated  voice. 

Some  pertinent  questions  arising  from  the  F-14  study  concerned:  a,  the 
distinctiveness  and  intelligibility  of  a  computer  generated  synthetic  voice 
in  the  aircraft  hearing  situation;  b,  whether  or  not  the  pilot  can  separate 
radio-transmitted  human  voice  messages  of  interest  from  simultaneous  synthe¬ 
tic  voice  messages;  and  c,  the  optimum  method  of  presenting  such  messages 
through  the  headset. 

An  exploratory  study  was  designed  to  investigate  these  questions.  The 
intent  was  to  examine  differing  methodologies  in  presenting  the  stimuli  used, 
and  to  ascertain  trends  in  the  subjects  performance  data. 

Future  studies  will  use  these  data  to  establish  definitive  design 
recommendations  for  incorporation  of  synthetic  voice  in  aircraft. 

II  PROBLEM 

A  crew  workload  problem  exists  in  today's  tactical  military  aircraft 
which  appears  to  contribute  to  mission  failure  and/or  loss  of  crew  and 
aircraft.  Investigations  point  to  crew  overloading  during  critical  mission 
phases,  primarily  in  the  visual  sensory  channel. 


Much  effort  has  been  expended  to  ease  crew  task  loading.  Individual 
control/display  developments •  such  as  the  head-up  display  or  the  multi¬ 
function  display,  are  orly  partial  solutions  since  they  have  not  reduced  the 
amount  of  information  input  to  the  crew,  and  have  generated  new  problems. 
These  problems  result  in  maintaining  a  high  loading  on  crew  visual  am)  motor 
sensory  channels. 

An  alternative  method  of  crew/system  interaction  is  needed.  This 
method  must  unburden  the  crew's  visual  and  motor  information  channels  (i.e. 
free  eyes  and  hands  for  other  tasks);  it  must  not  disrupt  attention;  it  must 
allow  quicker  assessment  of  situations  and  faster  decision  making,  which  in 
the  military  can  mean  the  difference  between  life  and  death  and  the 
conservation  or  loss  of  expensive  equipment. 

There  is  general  agreement  in  the  literature  that  performance  is 
enhanced  by  moving  some  of  the  crew's  information  input  from  the  visual 
channel  to  the  auditory  channel.  It  is  reasonable  to  assume  that  the  use  of 
a  voice  mode  should  be  equal  to  or  more  accurate  than  current  information 
presentation  methods,  and  should  allow  the  crew  greater  freedom  to  perform 
other  tasks.  By  using  the  auditory  sense  to  perceive  part  of  the  informa¬ 
tion  load,  eyes  and  hands  can  be  diverted  to  accommodate  other  inputs. 

A  synthetic  voice  capability  to  alert  the  flight  crew  to  cautions  and 
warnings  has  been  used  in  both  the  F-18  and  the  F-15  aircraft.  Preliminary 
information  tonds  to  support  the  contention  of  enhanced  crew  performance 
under  high  work-load  conditions. 

Unquestionably,  todays  voice  generation/computer  technology  allows  the 
use  of  speech  display  in  the  cockpit.  Equipment  is  available  for  the  gener¬ 
ation  of  several  types  of  speech,  e.g.,  female  voice,  male  voice  and  mechani 
cal  or  robotic  sounding  voice.  A  synthetic  female  voice  has  been  chosen  for 
use  in  both  the  F-18  and  F-15  aircraft,  but  experimental  data  supporting 
this  decision  is  lacking.  (Some  studies  mention  that  a  few  pilots  question 
the  use  of  a  female  voice). 
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When  the  synthetic  voice  is  transmitted  to  the  F-18  and  F-15  pilots 
through  the  intercom,  radio  reception  is  muted  to  the  degree  that 
essentially  it  is  turned  off.  This  method  precludes  the  simultaneous 
reception  of  a  potentially  important  radio  message,  as  for  example,  during 
air  combat.  The  major  investigation  of  this  study  is  whether  or  not  the 
synthetic  voice  message  and  the  radio  message  can  be  perceived  and 
understood  when  presented  simultaneously. 

A  literature  search  yielded  no  studies  on  the  discrimination  and  rela¬ 
tive  intelligibility  between  competing  synthetic  and  human  voice  messages. 
The  bulk  of  related  studies  on  hearing  simultaneous  or  overlapping  messages 
were  performed  between  the  early  1950’s  and  the  mid-1960’s.  It  was  hoped 
that  methods  could  be  discovered  which  would  alleviate  or  mitigate  the 
confusion  which  occurs  when  attempting  to  listen  to  and  respond  to  simultan¬ 
eous  or  overlapping  message  transmissions.  These  past  experiments  investi¬ 
gated  various  aspects  of  listening,  understanding  and  responding  to  one  or 
more  of  a  number  of  simultaneously  received  messages.  However,  they  did  not 
address  the  use  of  synthetic  voice,  did  not  consider  the  cockpit 
environment,  did  not  deal  with  short  messages  within  a  finite  set,  and  in 
most  cases  used  naive  subjects. 

Some  results  of  these  experiments,  however,  are  applicable  directly  to 
the  problem  of  discriminating  two  simultaneously  spoken  messages: 

1.  Each  voice  should  have  different  qualities,  i.e.,  pitch,  speed, 

male/  female,  accent,  intensity  level. 

2.  Each  voice  should  originate  from  an  apparent  different  direction. 

3.  Each  voice  should  involve  differing  syntax  and  deal  with  different 

subject  matter. 

4.  The  listener  should  be  trained  and  experienced  in  the  use  of 
pertinent  message  vocabulary,  syntax  and  context. 

For  our  application,  item  1  is  a  given  in  that  major  differences  do 
exist  between  synthetic  speech  and  human  speech.  These  differences  can  be 
enhanced  further  by  equipment  design,  selection  of  voice  type  and  by  the 
perceivable  difference  in  quality  between  radio  and  intercom. 
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Item  2  can  be  implemented  in  the  cockpit.  However,  investigation  Is 
required  to  determine  if  competing  messages  can  be  perceived  with  intelli¬ 
gibility.  Another  facet  of  this  question  is  how  the  simultaneously  received 
messages  (synthetic  voicA  and  human  voice)  should  be  presented  to  the  crew. 
The  manner  in  which  these  messages  are  presented  through  the  headset  will 
Impact  perception  and  intelligibility,  e.g.,  both  messages  to  both  ears,  each 
message  to  separate  ears,  etc. 

Item  3  also  is  implemented  easily  since  the  Inherent  differences  in  use 
requirements  between  radio  and  synthetic  speech  generation  equipment  involve 
differing  syntax  and  subject  matter. 

Item  A  will  be  an  inevitable  part  of  a  pilot's  training. 

Further  examination  of  the  literature  indicated  that  some  recent 
(1970's)  investigations  have  studied  individual  word  intelligibility  under 
stressful  flying  conditions  and  the  relative  alerting  capability  between 
tones  and  words  (taking  into  account  semantic  context).  However,  discrimina¬ 
tion  and  relative  intelligibility  between  competing  synthetic  and  human  voice 
messages  apparently  still  have  not  been  examined. 

Ill  OBJECTIVE 

This  experiment  was  designed  to: 

1 .  Determine  the  capability  to  recognise  and  understand  computer 
generated  female  voice  messages  in  the  presence  of  human  male  voice 
radio  messages. 

2.  Determine  the  capability  to  recognize  and  understand  human  male 
voice  radio  messages  of  interest  and  computer  generated  female 
voice  messages  when  presented  simultaneously. 

3.  Evaluate  the  effect  of  listening  method  on  message  discrimination 
and  intelligibility. 

A.  Evaluate  the  methodology  used  in  this  study. 

5.  Suggest  further  questions  for  Investigation. 
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IV  APPROACH 

A.  General 

Six  subjects,  all  experienced  flyers,  performed  an  aircraft  tracking 
task  while  listening  to  audio  tapis  of  typical  air  traffic  control  (ATC) 
radio  messages  and  a  synthetic  female  voice  enunciating  cautions  and  warn¬ 
ings.  Caution  and  warning  messages  were  selected  for  the  synthetic  voice 
messages  since  they  appear  obvious  candidates  for  cockpit  applications.  The 
F-18  and  F-15  aircraft  have  synthetic  voice  for  caution  and  warning  messages 
in  flight  test  aircraft.  The  same  application  was  recently  studied  for  use 
in  the  Grumman  F-14  aircraft. 

B.  Tracking  Task 

The  tracking  task  was  performed  by  the  subject  as  a  symbolic  represen¬ 
tation  of  flying  an  airplane.  A  horizon  line  and  aircraft  symbol  were 
displayed  on  the  CRT.  The  force  stick  controller  commanded  aircraft  symbol 
movement.  The  computer  was  programmed  to  move  the  aircraft  symbol  in  such  a 
manner  that  the  subject  had  to  continually  manipulate  the  control  to  main¬ 
tain  level  flight.  Information  was  presented  to  the  subject  on  a  CRT  to 
simulate  a  flying  task.  The  words  ELEV  and  ANGLE  appeared  whenever  the 
subject  allowed  the  aircraft  symbol  to  shift  1/8"  above  or  below  the  horizon 
line  and/or  to  deviate  by  ±3°  in  roll  from  the  horizontal.  These  data 
simulated  the  information  a  pilot  gathers  from  his  flight  instrumentation 
during  flight.  The  graphic  image  simulated  his  attitude  indicator  (Figure 
1).  Aircraft  control  was  maintained  with  a  force  stick  hand  controller. 

The  computer  stored  the  following  data  which  was  printed  out  after  each 
run:  a,  distance  and  angle  off-track;  b,  on-track  time;  c,  off-track  time; 
and  d,  cumulative  total  of  off-track  occurances. 
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HORIZON  LINE 


FIGURE  1:  TRACKING  TASK  CRT  DISPLAY 


C .  Experiment  Description 

Each  subject  was  requested  to  attend  to  radio  messages  preceded  by  his 
assigned  call  sign  and  to  repeat  them  ou  .  ^.oud  while  performing  the  tracking 
task.  The  subjects  also  were  requested  to  attend  to  all  caution  and  warning 
messages  and  repeat  them  out  loud.  Further,  they  were  told  that  radio 
messages  and  caution  and  warning  messages  could  be  presented  simultaneously, 
and  that  they  should  repeat  both.  The  subjects  were  instructed  that,  at  tne 
very  least,  they  should  state  aloud  the  sense  of  what  was  understood  if  they 
could  not  repeat  the  messages  verbatim. 

Three  radio  messages  with  the  subject's  call  sign  preceding  the  message 
and  four  caution  and  warning  messages  were  presented  on  each  of  the  three 
experiment  runs. 

It  was  emphasized  that  the  aircraft  attitude  and  altitude  must  be 
maintained  at  all  times. 

All  verbal  responses  were  recorded. 

Each  subject  performed  the  tracking  task  and  was  presented  with 
messages  to  repeat  three  separate  times.  He  was  not  informed  about  the 
different  methods  of  message  presentation.  Each  session  was  characterized 
by  a  different  method  of  message  presentation  through  the  earphones: 

Method  I:  DIOTIC  -  Radio  messages  and  caution  and  warning  messages 

were  presented  to  both  ears  simultaneously. 

Method  II:  DIOTIC/DICHOTIC  -  The  cautions  and  warnings  were  presented 

to  one  ear  while  the  radio  messages  were  presented  to  both  ears. 

Method  III:  DICHOTIC  -  Radio  messages  were  presented  to  both  ears. 
During  simultaneous  transmission  of  the  radio  message  and  the  caution 
and  warning  message,  the  radio  message  was  blanled  in  the  ear  receiving 
the  caution  and  warning  message. 

Figure  2  depicts  these  conditions.  Table  1  presents  the  matrix  of 
conditions  used  in  the  study.  It  should  be  noted  that  the  presentation  order 
to  the  subjects  of  the  *  :.ree  message  conditions  was  counterbalanced.  This 
ordering  precludes  carryover  effects  that  might  occur  from  one  experiment  run 
to  another,  thereby  influencing  a  subject's  performance. 
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FIGURE  2:  LISTENING  METHODS.  The  manner  of  presenting 
simultaneously  received  synthetic  and  human 
voices  through  the  headphones 
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TABLE  1:  STUDY  MATRIX* 


LISTENING 

METHOD 

RADIO  MESSAGES 

SYNTHETIC  MESSAGES 

SUBJECT 

SUBJECT 

A  B  C  D  E  F 

A  B  C  D  E  F 

1 

1 

DIOTIC 

2 

2 

3 

BOTH  EARS 

4 

BOTH  EARS 
SUPERIMPOSED 

4 

5 

DIOTIC 

5 

6 

DICHOTIC 

6 

7 

BOTH  EARS 

8 

ONE  EAR 
SUPERIMPOSED 

7 

9 

8 

10 

OICHOTIC 

9 

11 

BOTH  EARS 

12 

ONE  EAR 

RADIO  BLANKED 

*  As  shown,  in  some  cases  messages  were  presented 
alone,  that  is,  without  overlap  or  synchrony  with 
the  other  type  -  radio  vs.  synthetic  or  synthetic 
vs.  radio.  This  was  to  allow  for  the  determination 
of  the  subject's  perception  without  the  masking  or 
confusing  possibility  presented  when  messages  are 
received  simultaneously. 
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V. 


APPARATUS 


The  study  was  performed  in  the  Grumman  Plant  14  Advanced  Cockpit  Tech¬ 
nology  Lab  (ACTL) .  A  CRT  display  and  force  stick  hand  controller  were  used 
for  a  tracking  task  designed  to  preclude  the  subject  from  concentrating 
solely  on  his  listening  task.  The  Lab's  Varian  76  computer  was  programmed 
to  provide  a  fixed  horizon  and  moving  aircraft  symbol  on  the  Sanders 
ADDS  900  Graphics  System  CRT  which  the  subject  commanded  via  the  force  stick 
control. 

Audio  cassette  tapes  were  played-back  on  two  JVC  DD-5J  stereo  cassette 
decks.  Both  subject  and  experimenter  listened  to  the  tapes  through  Panasonic/ 
Technics  EAH-810  stereo  headsets.  The  subject’s  voice  was  picked  up  by  a 
microphone  for  the  recording  of  his  responses  on  a  portable  cassette  recorder. 
The  subject's  headset  and  microphone  were  plugged  into  a  junction  box.  The 
experimenter's  headset  was  plugged  into  a  specially  designed  audio  mixer 
which  controlled  the  routing  of  the  taped  messages  to  the  subject  via  the 
junction  box  and  directly  to  the  experimenter  (Figure  3).  The  experiment 
set-up  is  shown  in  Figures  4,  5  and  6. 

VI  VOICE  STIMULI  TAPES 

A.  Generation  of  Radio  Messages 

Air  Traffic  Control  (ATC)  radio  messages  were  obtained  by  setting  up  a 
radio  receiver  tuned  to  local  air  space  frequencies,  tying  in  a  voice 
operated  circuit  and  recording  enough  messages  to  fill  up  an  audio  cassette 
with  more  than  eight  hours  of  messages  compressed  into  one  hour  of  tape.  A 
sufficient  number  of  messages  with  a  particular  call  sign  were  gleaned  from 
this  tape  for  use  in  the  experiment. 

Selected,  reasonably  intelligible,  messages  were  chosen  from  this 
source  tape  for  replay  to  the  subject.  Messages  which  did  not  require  a 
response  by  the  subject  were  interspersed  among  those  messages  of  interest, 
which  required  the  subject  to  respond.  Silent  periods  as  well  as  un-intel- 
ligible  voices  also  were  used  between  intelligible  voices  to  simulate  air 
traffic  control  and  in-flight  communications. 
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B.  Generation  of  Synthetic  Voice  Messages 


A  tape  was  obtained  from  a  manufacturer  of  synthetic  voice  apparatus. 
This  tape  contained  the  set  of  caution  and  warning  messages  tailored  for  the 
F-14  aircraft.  These  messages  were  re-recorded  for  re-play  in  synchrony 
with  the  radio  messages. 

C.  Tapes  Used  In  Study 

Three  tapes  were  produced  using  the  source  ATC  radio  communication 
tape,  one  tape  for  each  listening  method.  Since  nine  messages  were  required 
and  six  were  available,  three  were  used  twice  each.  Similarly,  three  tapes 
using  selected  synthetic  voice  messages  were  produced  from  the  single 
vendor's  tape.  Twelve  different  messages  were  chosen  from  a  total  of 
fifteen  available.  The  tapes  chosen  for  use  during  the  DICHOTIC  condition 
(both  radio  and  synthetic  voices)  were  combined  into  one  tape  for  replay  to 
the  subject.  Since  it  was  not  feasible  to  use  an  actual  voice  operated 
circuit  to  interrupt  the  radio  voice  in  one  ear,  a  reel-to-reel  recorder  was 
used  which  allowed  separate  recording  on  each  of  two  tracks.  The  radio 
messages  were  recorded  on  both  tracks  of  a  reel-to-reel  tape,  then  the 
synthetic  voice  messages  were  interjected  at  the  appropriate  points  on  one 
of  the  two  tracks.  This  recording  was  transferred  to  a  single  cassette  for 
use  in  the  study. 

I •  Audio  Levels  of  Radio  Messages  and  Synthetic  Voice  Cautions  and  Warnings 

As  previously  stated,  source  tapes  were  re-transcribed  on  cassettes  for 
use  in  the  experiment.  Since  the  ATC  radio  messages  varied  in  audio  level 
depending  upon  the  distance  and  power  levels  of  the  original  source,  a 
subjective  evaluation  was  made  during  the  re-taping  to  equalize  the  audio 
levels  of  all  radio  messages  of  interest. 

Similarly,  a  subjective  evaluation  was  made  on  the  synthetic  voice 
caution  and  warning  (C&W)  messages  to  equalize  their  levels  with  the  radio 
levels.  However,  since  the  caution  and  warning  messages  were  not 
radio-transmitted,  but  generated  by  a  computer  and  directly  recorded,  they 
sounded  stronger  even  at  audio  level  settings  which  equaled  the  radio 
settings.  This  is  attributed  to  a  greater  signal-to-noise  (S/N)  ratio 
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inherent  in  the  method  of  generation.  Discussions  with  pilots  before  the 
experiment  and  the  subjects  after  the  experiment  verified  that  this 
difference  in  audio  levels  between  radio  and  synthetic  voice  caution  and 
warning  compares  with  the  difference  in  audio  levels  between  radio  and 
intercom  in  an  aircraft.  It  appears  then  that  the  audibility  levels  of  the 
synthetic  voice  messages  and  the  radio  voice  messages  were  reasonably 
reproduced  in  the  study. 

VII  TEST  SUBJECTS 

Six  test  subjects  were  randomly  selected  from  a  population  of  experi- 
enced  pilots  in  the  employ  of  the  Grumman  Aerospace  Corporation.  These 
employees  hold  current  pilots  licenses  and/or  have  military  flying  experi¬ 
ence.  They  passed  an  audiomctric  screening  performed  by  the  Grumman  Medical 
Department . 

VIII  PROCEDURE 

Each  subject  was  tested  individually  to  preclude  discussion  of  the  study 
procedure  with  the  other  subjects.  Each  received  a  pre-test  briefing. 
Written  instructions  were  given  to  the  subjects  to  ensure  that  each  received 
identical  directions.  See  Appendix,  page  30.  Each  subject  received  a 
familiarization  run  to  become  aqualnted  with  the  tracking  task  and  messages 
to  be  received.  He  also  practiced  responding  to  the  messages.  During  the 
familiarization  run,  both  radio  and  synthetic  voice  messages  were  presented 
separately  (not  simultaneously)  and  to  both  ears  (diotically) .  The  radio 
messages  presented  were  typical  ATC  conversations  which  included  the  same 
messages  to  be  received  during  the  experiment.  The  synthetic  voice  and  the 
synthetic  caution  and  warning  messages  also  were  the  same  as  used  during  the 
experiment.  A  list  of  these  synthetic  voice  messages  was  given  to  each 
subject  during  the  familiarization  period.  The  rationale  for  providing  this 
message  list  was  that  a  pilot  who  flies  a  military  aircraft  incorporating  a 
synthetic  voice  caution  and  warning  system,  through  intensive  training,  will 
be  intimately  familiar  with  the  limited  vocabulary  and  the  voice  itself. 
Conversely,  a  pilot  does  not  know  precisely  what  he  will  hear  over  the  radio. 

The  familiarization  run  was  complete  when  the  subjecc  had  listened  to 
the  message  tape  at  least  once  and  was  satisfied  with  his  handling  of  the 
force  stick  controller. 
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Each  subject  experienced  three  separate  experiment  runs.  As  earlier 
indicated ,  each  run  was  characterized  by  a  different  manner  of  auditory 
presentation  of  the  radio  and  synthetic  voice  messages  to  the  subject's 
ears.  Table  2  presents  the  randomized  order  of  presentation  of  these 
messages. 

After  completion  of  all  three  runs  for  each  subject ,  a  debriefing 
between  experimenter  and  subject  was  held.  First,  a  questionnaire  was  com¬ 
pleted  by  the  subject  to  elicit  his  subjective  evaluations  of  the  quality  of 
both  the  synthetic  voice  and  the  radio  voice.  A  copy  of  the  questionnaire 
appears  in  the  Appendix  on  pages  31  and  32.  Then,  a  free-ranging  discussion 
was  held  covering  the  questionnaire  and  the  subject's  comments,  opinions  and 
suggestions  concerning  airborne  voice  communications  based  on  hJs  flight 
experience. 

IX  PERFORMANCE  MEASURES 

The  subject  was  required  to  repeat  each  message  aloud  when  he  perceived 
a  "message  of  interest".  Messages  of  interest  are  defined  as:  a,  all 
synthetic  voice  messages;  and  b,  all  radio  messages  preceded  by  I  he  key  call 
sign  "94  WI.ISKEY".  Scoring  was  performed  by  the  experimenter  listening  to 
the  subject  and  determining  if  the  message  was  repeated  or  responded  to 
correctly  at  the  right  time.  The  subject's  verbal  response  was  recorded  for 
later  verification  and  evaluation. 

X  RESULTS 

A.  Performance  Indices 

Test  subject  response  data  weie  collected  to  indicate  perception  aad 
understanding  of  the  audio  stimuli  presented  in  each  of  the  three  listening 
methods.  These  data  are  presented  ii.  the  form  of  performance  indices.  The 
indices,  expressed  as  percentages,  have  been  converted  from  ratios  derived 
from  the  series  of  test  subject  responses  collected  during  the  experiment. 
They  are  used  ur,  measures  of  the  test  subject's  ability  to  perceive  and 
understand  both  radio  and  synthetic  voice  messages. 
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Respond  is  were  collected  in  three  categories:  a,  perceived  and  under¬ 
stood;  b,  perceived  but  not  understood;  and  c,  not  perceived  (therefore 
not  possible  to  understand) . 

To  enable  quantification  of  the  data,  numerical  scores  were  assigned  to 
the  collected  responses,  thusly: 

Message  perceived  and  understood  *  10  points 

Message  perceived  but  not  understood  ■  5  points 

Message  not  perceived  *  0  points 

The  performance  index,  expressed  as  a  percentage,  was  derived  by  divid¬ 
ing  the  actual  score  by  the  possible  total  score.  For  example,  out  of  a 
total  of  six  messages  transmitted,  if  three  messages  were  understood,  two 
messages  not  perceived,  and  one  message  perceived  but  not  understood, 
scoring  would  be  10  +  10  +  10  +  0  +  0  +  5  respectively.  The  actual  score 
then,  would  be  expressed  as  35/60.  The  denominator  60  results  from  the 
total  number  of  six  transmitted  messages  times  10  (total  possible  score). 
Conversion  to  a  percentage  gives  an  index  of  58.3%. 

B.  Tracking  Task 

Since  the  tracking  task  was  used  only  to  keep  the  subject  busy  so  he 
would  not  concentrate  on  listening  for  the  audio  messages,  an  in-depth 
analysis  of  the  computerized  tracking  data  was  not  performed.  However,  a 
perusal  of  these  data  indicated  no  apparent  decrement  in  performance  during 
the  periods  when  the  subject's  attention  was  shared  with  pertinent  radio 
and/or  caution  and  warning  messages. 

C.  Effects  of  Listening  Methods 

The  raw  response  data  were  converted  to  numerical  scores  as  shown  in 
Figure  7.  Inspection  of  the  data  for  listening  methods  indicates  that 
synthetic  voice  caution  and  warning  messages  generally  produced  a  much 
higher  percentage  of  correct  responses  than  did  radio  messages.  This  result 
is  not  suprising  since  the  quality  of  the  synthetic  voice  messages  is 
significantly  better  than  typical  aircraft  radio  messages. 
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A  review  of  the  data  for  all  radio  messages  (Figure  7A)  compared  with 
all  synthetic  voice  caution  and  warning  messages  (Figure  7B) ,  as  plotted  In 
Figure  8,  shows  only  slight  differences  (between  listening  methods)  in  the 
percentage  of  correct  responses  of  either  radio  messages  or  synthetic  voice 
caution  and  warning  messages.  However,  when  the  data  are  separated  into 
categories  of  simultaneous  reception  of  competing  messages  (Figure  9),  and 
reception  with  no  competition  (Figure  10),  large  differences  appear. 

The  data  in  Figure  9  show  that  the  method  of  message  presentation 

affects  understanding.  Apparently,  the  use  of  a  dlchotlc  method  for 

presenting  a  caution  and  warning  message  will  permit  greater  understanding 
of  the  radio  message  with  only  a  slight  detriment  to  the  understanding  of 
the  synthetic  voice  message. 

The  data  in  Figure  10  show  expected  results.  When  either  type  of 

message  is  received  with  no  simultaneous  competition,  it  will  be  understood. 

D.  Effects  of  Masking  the  Radio  Call  Sign 

It  should  be  remembered  that  the  subjects  were  instructed  to  respond 
only  to  radio  messages  that  were  preceded  by  a  specific  call  sign. 

Therefore,  any  message  not  preceded  by  that  call  sign  would  be  ignored.  It 
is  obvious  that  the  perception  of  the  call  sign  is  vital  to  the  subject's 
response.  Thus,  an  analysis  was  performed  on  subject  performance  as  affected 
by  the  degree  of  obfuscation  of  the  radio  message  call  sign  by  the  synthetic 
voice  message. 

A  review  of  Tables  3,  4  and  5  indicates  the  overlap  of  the  caution  and 
warning  and  radio  messages  when  simultaneously  presented  to  the  subject. 

Tables  6  and  7  show  that  where  the  call  sign  was  presented  from 
partially  to  fully  obscured,  more  and  more  messages  were  missed.  Obviously, 
the  obscuring  of  the  call  sign  by  the  synthetic  voice  messages,  as  had 
occurred  in  the  study,  could  be  a  real-life  situation.  However,  the 
probability  of  such  occurrances  are  considered  to  be  low.  Therefore,  it 
could  be  expected  that  a  pilot  would  be  capable  of  perceiving  the  majority 
of  the  radio  messages  directed  to  him. 


20 


FIGURE  8.  PERCENT  CORRECT  RESPONSES  FOR  EACH  LISTENING  METHOD. 
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FIGURE  9.  PERCENT  CORRECT  RESPONSES  FOR  KEY  RkDIO  MESSAGES  AND  SYNTHETIC 
VOICE  CAUTION  AND  WARNING  MESSAGES  DURING  SIMULTANEOUS 
RECEPTION. 


FIGURE  10.  PERCENT  CORRECT  RESPONSES  FOR  KEY  RADIO  MESSAGES  AND  SYNTHETIC 
VOICE  CAUTION  AND  WARNING  MESSAGES  WHEN  RECEIVED  INDIVIDUALLY 
WITH  NO  SIMULTANEOUS  COMPETITION. 


TABLE  7:  COMPARISONS  OF  RADIO  MESSAGE  PERCEPTION  BY 
CATEGORY  OF  CALL  SIGN  OBFUSCATION 


DEGREE  OF  CALL  SIGN  OBFUSCATION 

CLEAR 

PARTIAL 

OBSCURED 

TOTALS 

NO.  MESSAGES  MISSED 

2 

6 

13.5 

21.5 

NO.  MESSAGES  UNDERSTOOD 

22 

9 

1.5 

32.5 

E .  Understanding  Radio  Messages  With  Unobstructed  Call  Signs 


A  further  examination  of  Tables  3,  4  and  3  reveals  that  during  simul¬ 
taneous  reception,  the  synthetic  voice  message  affected  the  subject's 
perception  of  the  radio  message  differently  depending  upon  the  relative 
onset  of  each  message. 

When  both  messages  started  simultaneously,  the  message  with  greater 
audibility  (synthetic  voice)  obscured  the  radio  message  call  sign. 
Therefore,  even  though  the  subject  may  have  perceived  the  message  following 
the  call  sign,  he  did  not  know  that  it  was  directed  to  him.  He  did  not  pay 
attention  and  the  message  was  lost. 

When  the  synthetic  voice  message  started  after  the  radio  call  sign, 
most  subjects  were  able  to  perceive  both  messages.  Most  of  these  subjects 
were  able  to  understand  the  synthetic,  voice  message,  while  some  also  were 
able  to  understand  the  radio  message.  Apparently,  those  subjects  who 
exhibited  this  dual  understanding  were  able  to  switch  attention  quickly  from 
one  message  to  another.  Also,  since  they  could  not  perceive  the  radio 
message  words  at  the  same  time  as  they  attended  to  the  synthetic  voice,  it 
appears  likely  that  the  meaning  of  the  lost  words  were  reconstructed  from 
the  context  of  the  message  portion  that  was  understood.  (Discussion  during 
debriefing  tends  to  support  this  thesis). 

A  questionnaire  eliciting  subjective  evaluations  and  comments  was 
completed  by  each  subject  immediately  following  the  finish  of  his  test  runs. 
A  copy  of  each  completed  questionnaire  is  included  in  the  Appendix  (pages  33 
to  44).  In  all  cases  the  quality  of  both  radio  and  synthetic  voices  used  in 
this  study  were  rated  satisfactory  or  better. 
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XII  CONCLUSIONS 


This  preliminary  study  has  permitted  insight  into  four  basic  events: 

1.  Key  radio  messages  can  be  heard  and  understood  when  presented 
simultaneously  with  synthetic  female  voice  caution  and  warning 
messages . 

2.  It  is  essential  that  the  subject  perceive  his  call  sign  if  he  is 
to  hear  and  understand  the  radio  message. 

3.  Virtually  all  synthetic  female  voice  caution  and  warning  messages 
can  be  heard  and  understood,  whether  or  not  any  radio  messages  are 
present . 

4.  There  are  differences  in  the  ability  of  a  subject  to  hear  and 
understand  radio  messages  as  a  function  of  different  methods  of 
presenting  competing  synthetic  female  voice  caution  and  warning 
messages . 

The  results  of  thi^  study  tend  to  support  the  belief  that  synthetic 
voice  messages  (in  this  case,  female  voice)  provide  an  effective  means  of 
pre  nting  alerting  messages  and  data  to  a  pilot. 

Additionally,  it  should  not  be  necessary  to  mute  radio  messages  to  a 
1  below  the  pilot's  perception  as  is  done  in  at  least  two  aircraft 
to  ",  thereby  ensuring  the  loss  of  radio  messages  during  the  transmission 
of  synthetic  voice  messages. 

F  rther,  it  appears  that  some  radio  messages  may  be  understood  even 
though  a  part  of  the  message  may  be  lost  to  the  pilot's  understanding. 
Appa.ntly,  the  sense  of  the  message  is  reconstructed  from  the  context  of 
the  message  portion  that  is  understood. 

Future  studies  are  planned  to  investigate  further  these  findings.  It 
is  intended  to  include  the  investigation  of  other  types  of  synthetically 
generated  voices,  such  as  natural  male  and  robotic  or  neuter.  Particular 
attention  will  be  paid  to  reaffirming  the  efficacy  of  the  dichotic  listening 
method . 
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TEST  SUBJECT  INSTRUCTIONS 


This  experiment  in  which  you  are  participating  is  designed  to  evaluate 
cockpit  communications  and  aircraft  attitude  control.  Your  function  is  that 
of  an  experienced  pilot  flying  your  aircraft  and  responding  to  pertinent 
communications . 

You  will  be  seated  in  front  of  a  large  CRT  and  you  will  wear  headphones. 

A  microphone  will  be  placed  in  front  of  you.  A  forcestick  control  will  be 
adjacent  to  the  CRT.  It  may  be  placed  wherever  you  wish  for  your  comfort. 

A  graphic  representation  of  an  attitude  control  instrument  (containing  a 
horizon  line  and  aircraft  symbol)  will  be  presented  on  the  CRT.  luring  the 
experiment  your  job  will  be  to  maintain  aircraft  attitude  by  use  of  the  force- 
stick  control.  You  will  be  controlling  the  .Aircraft  symbol  movement.  In 
addition,  the  words  ELEV  and  ANGLE  will  appear  in  the  upper  corners  of  the 
CRT.  This  additional  information  will  appear  whenever  you  are  not  straight 
and  level. 

Two  types  of  comm  messages  will  require  your  responses: 

1.  Whenever  you  hear  a  caution  or  warning  message,  you  are  to  repeat  it 
aloud. 

2.  Whenever  you  hear  a  message  with  the  call-sign  "94  WHISKEY",  you  are 
to  repeat  it  aloud. 

Should  you  hear  a  C&W  message  and  a  radio  message  at  about  the  same  time, 
you  are  to  repeat  them  aloud.  YOU  SHOULD  DELAY  YOUR  VERBAL  RESPONSE  TO  ENSURE  HEARING 
THE  ENTIRE  MESSAGE (S).  In  the  event  you  hear  a  message (s)  as  described  above, 
but  do  not  catch  all  the  words,  state  aloud  what  you  think  the  message  was  about. 

You  cannot  request  a  repeat  of  the  message. 

Remember,  you  must  maintain  aircraft  attitude  control  at  all  times,  other¬ 
wise  you  risk  departure  from  normal  flight. 

Before  donning  headset,  check  for  right/left  orientation. 
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TEST  SUBJECT  BATING  SHEET 


NAME: 


Please  check  the  bos  adjacent  to  your  selection  for  each  statement. 

1.  Rate  the  quality  of  rynthetlc  voice  in  this  study. 

O  Extremely  Poor  O  Poor  O  Satisfactory  Q  Good  O  Excellent 

2.  Rate  your  ability  to  perceive  the  synthetic  voice  messages. 

O  Extremely  poor  CD  Poor  □  Satisfactory  CD  Good  Q  Excellent 

3.  Rate  your  ability  to  understand  the  synthetic  voice  messages. 

O  Extremely  poor  QPoor  CD  Satisfactory  D  Good  □  Excellent 

£ 

4.  Rate  the  quality  of  radio  voice  in  the  cockpit  during  actual  flight. 

CD  Extremely  poor  QPoor  D  Satisfactory  D  Good  Q  Excellent 

5.  Rate  the  quality  of  radio  voice  in  this  study. 

CD  Extremely  poor  CD  Poor  □  Satisfactory  CD  Good  Q  Excellent 

6.  Rate  your  ability  to  perceive  your  radio  call  sign. 

□  Extremely  poor  Q  Poor  □  Satisfactory  CD  Good  O Excellent 

7.  Rate  your  ability  to  understand  the  radio  message  following  your  call  sign. 

H  Extremely  poor  O  Poor  CD  Satisfactory  O  Good  O  Excellent 
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TEST  SUBJECT  RATING  SHEET  (Cont*d.1 


8.  Did  you  encounter  any  problems  in  attending  to  and/or  understanding 
simultaneous  messages  (synthetic  and  radio  voice)? 


□  Yes 

□  No 


If  your  answer  is  Yes,  please  elaborate: 


Use  the  space  below  for  providing  additional  comments  and/or 
evaluations. 


TEST  SUBJECT  RATING  SHEET 


NAME: 


A 


Please  check  the  box  adjacent  to  your  selection  for  each  statement. 

1.  Rate  the  quality  of  synthetic  voice  in  this  study. 

O  Extremely  Poor  O  Poor  (^Satisfactory  Q  Good  O  Excellent 

2.  Rate  your  ability  to  perceive  the  synthetic  voice  messages. 

O  Extremely  poor  0  Poor  Gfsatisfactory  CD  Good  Q  Excellent 

i 

3.  Rate  your  ability  to  understand  the  synthetic  voice  messages. 

0  Extremely  poor  GJpoor  □  Satisfactory  □  Good  □  Excellent 

A.  Rate  the  quality  of  radio  voice  in  the  cockpit  during  actual  flight. 

0  Extremely  poor  0  Poor  STsatisfactory  O  Good  Q  Excellent 

5.  Rate  the  quality  of  radio  voice  in  this  study. 

O  Extremely  poor  0Poor  0  Satisfactory  OTbood  O  Excellent 

6.  Rate  your  ability  to  perceive  your  radio  call  sign. 

O  Extremely  poor  OPoor  0  Satisfactory  CfGood  0  Excellent 

7.  Rate  your  ability  to  understand  the  radio  message  following  your  call  sign. 

O Extremely  poor  OPoor  [^Satisfactory  0  Good  O Excellent 
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Use  the  space  below  for  providing  additional  comments  and/or 
evaluations. 


Prtpft.rej'i/m  For  }  <*>c. 


« 


34 


TEST  SUBJECT  RATING  SHEET 


Please  check  the  box  adjacent  to  your  selection  for  each  statement, 


1.  Rate  the  quality  of  synthetic  voice  in  this  study. 

O  Extremely  Poor  O  Poor  ©'Satisf actory  Q  Good  □  Excellent 

2.  Rate  your  ability  to  perceive  the  synthetic  voice  messages. 

O Extremely  poor  O  Poor  O Satisfactory  □  Good  QJjJxcellent 


3.  Rate  your  ability  to  understand  the  synthetic  voice  messages. 
0  Extremely  poor  Q  Poor  O Satisfactory  (^Good  □ 


Excellent 


4.  Rate  the  quality  of  radio  voice  in/the  cockpit  during  actual  flight. 


0  Extremely  poor 


radio  voice  iiyt 
Q  Poor  ©Sat 


Satisfactory  0  Good  0  Excellent 


5.  Rate  the  quality  of  radio  voice  in  this  study. 


O  Extremely  poor  O  Poor  0  Satisfactory  (JflGood  0  Excellent 


Qd  Goo< 


Rate  your  ability  to  perceive  your  radio  call  sign. 

/ 

0 Extremely  poor  O Poor  O Satisfactory  O  Good  EEjExcellent 


Rate  your  ability  to  understand  the  radio  message  following  your  call  sign. 
0 Extremely  poor  O Poor  O Satisfactory  (&  Good  o  Excellent 
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TEST* SUBJECT  RATING  SHEET  (Cont'd. 


Did  you  encounter  apy  problems  in  attending  to  and/or  understanding 
simultaneous  messages  (synthetic  and  radio  voice)? 

CD  Yes 

v_  ^  If  your  answer  is  Yes,  please  elaborate: 

OfNo 


Use  the  space  below  for  providing  additional  comments  and/or 
evaluations. 

Wo*.  -  —  c  ^  (J 

b  •  -7 

.Scr***  V 


e-  f  v  7\ 


i  \.  '  6  V  ?  H  ^ 


0  c<c  l{  ° 


)3‘-?a  /V 


£  ( C  £1  r^i  cc_. 

r«je,s  y  ^  /"1  e  * 

.  t,  a  //  f  ft  cjx  i  e  ^  ~  ^ 
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TEST  SUBJECT  RATING  SHEET 


NAME: 


Please  check  the  box  adjacent  to  your  selection  for  each  statement. 

1.  Rate  the  quality  of  synthetic  voice  in  this  study. 

□  Extremely  Poor  O  Poor  0  Satisfactory  G^Good  0  Excellent 

2.  Rate  your  ability  to  perceive  the  synthetic  voice  messages. 

O  Extremely  poor  D  Poor  0  Satisfactory  D  Good  0'Excellent 

3.  Rate  your  ability  to  understand  the  synthetic  voice  messages. 

□  Extremely  poor  □  Poor  O  Satisfactory  S'Good  0  Excellent 

4.  Rate  the  quail  y  of  radio  voice  in  the  cockpit  during  actual  flight. 

0  Extremely  poor  □  Poor  O  Satisfactory  O^Good  0  Excellent 

5.  Rate  the  quality  of  radio  voice  in  this  study. 

□  Extremely  poor  O  Poor  Q^Satisf  actory  O  Good  0  Excellent 

6.  Rate  your  ability  to  perceive  your  radio  call  sign. 

□  Extremely  poor  □  Poor  Q^Sa^isf  actory  O  Good  o  Excellent 

7.  Rate  your  ability  to  understand  the  radio  message  following  your  call  sign. 

f~l Extremely  poor  O  Poor  Q'lsatisf  actory  O  Good  □  Excellent 

&  "TH<5  \fo^e  IW  Caul  i bu 
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_ iiiat»agi^aiaa£&: 


TEST  SUBJECT ‘RATING  SHEET  (Cont'd.) 


8.  Did  you  encounter  qny  problems  in  attending  to  and/or  understanding 
simultaneous  messages  (synthetic  and  radio  voice) ? 

Q  Yes 

If  your  answer  is  Yes,  please  elaborate: 


Use  the  space  below  for  providing  additional  comments  and/or 
evaluations . 


CQ  v*°  eVe*-3 T  of  ^iMot,'pA/j«7xos  "T-KT 

CAo-Q0»/ f€Y>. 


t 
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TEST  SUBJECT  RATING  SHEET 


NAME: 


Please  check  the  box  adjacent  to  your  selection  for  each  statement. 

1.  Rate  the  quality  of  synthetic  voice  in  this  study. 

□  Extremely  Poor  □  Poor  0  Satisfactory  □  Good  □  Excellent 

2.  Rate  your  ability  to  perceive  the  synthetic  voice  messages. 

□  Extremely  poor  0  Poor  □  Satisfactory  0  Good  □  Excellent 

3.  Rate  your  ability  to  understand  the  synthetic  voice  messages. 

□  Extremely  poor  0Poor  0  Satisfactory  □  Good  □  Excellent 

A.  Rate  the  quality  of  radio  voice  in  the  cockpit  during  actual  flight. 

□  Extremely  poor  0  Poor  0  Satisfactory  □  Good  0  Excellent 

5.  Rate  the  quality  of  radio  voice  in  this  study. 

□  Extremely  poor  O  Poor  0  Satisfactory  0  Good  o  Excellent 

6.  Rate  your  ability  to  perceive  your  radio  call  sign. 

□  Extremely  poor  O Poor  O  Satisfactory  0  Good  O Excellent 

7.  Rate  your  ability  to  understand  the  radio  message  following  your  call  sign. 

Q Extremely  poor  0  Poor  □  Satisfactory  0  Good  □  Excellent 
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!  AuSiwS  mi4 


TEST  *  SUBJECT  RATING  SHEET  (Cont'd.) 


8.  Did  you  encounter  aqy  problems  in  attending  to  and/or  understanding 
simultaneous  messages  (synthetic  and  radio  voice)? 

0Yes 

If  your  answer  is  Yes,  please  elaborate: 

.-»~p  a>op  o's.  cc-i  P)A^rt>  Arn«  s^zjc^io/Oi/ocr-  cv  -fw a:  . 


/Left/  C4“° 

/  (Word) 


6«u/ 


dvi<r&T£"MF 


A 

C* 


Use  the  space  below  for  providing  additional  comments  and/or 
evaluations. 


4 


40 


TEST  SUBJECT  RATING  SHEET  ' 


NAME: 


Please  check  the  box  adjacent  to  your  selection  for  each  statement. 


1.  Rate  the  quality  of  synthetic  voice  in  this  study. 


□  Extremely  Poor 

0  Poor 

^J^Satisf  actory 

0  Good 

0  Excellent 

2. 

Rate  your  ability 

to  perceive  the  synthetic  voice  messages 

• 

O  Extremely  poor 

J^Poor 

0  Satisfactory 

0  Good 

O  Excellent 

3. 

Rate  your  ability 

to  understand  the  synthetic  voice  messages. 

O Extremely  poor 

^'Poor 

O  Satisfactory 

O  Good 

O  Excellent 

4. 

Rate  the  quality  of  radio  voice  in  the  cockpit 

during  actual  flight. 

O Extremely  poor 

O  Poor 

^Satisfactory 

O  Good 

0  Excellent 

5. 

Rate  the  quality  of  radio  voice  in  this  study. 

O Extremely  poor 

0  Poor 

^/Satisfactory 

0  Good 

O  Excellent 

6. 

Rate  your  ability 

to  perceive 

your  radio  call 

✓ 

sign. 

O  Extremely  poor 

O  Poor 

^□‘Satisfactory 

0  Good 

O  Excellent 

7. 

Rate  your  ability 

to  understand  the  radio  message  following  your  call 

0  Extremely  poor 

^Jpoor 

0  Satisfactory 

O  Good 

CD  Excellent 

41 


TEST  SUBJECT  RATING  SHEET  (Cont*d.) 


8.  Did  you  encounter  apy  problems  in  attending  to  and/or  understanding 
simultaneous  messages  (synthetic  and  radio  voice)? 

0Yes 

If  your  answer  is  Yes,  please  elaborate 

□  No 

TO  /$TC.  Stfouco 

Cox^dsC  r/TT^-  c^/WF) 


Use  the  space  below  for  providing  additional  comments  and/or 
evaluations. 


« 
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TEST  SUBJECT  RATING  SHEET 


NAME: 


Please  check  the  box  adjacent  to  your  selection  for  each  statement. 


1.  Rate  the  quality  of  synthetic  voice  in  this  study. 

O  Extremely  Poor  □  Poor  □  Satisfactory  Q^ood  O  Excellent 

2.  Rate  your  ability  to  perceive  the  synthetic  voice  messages. 

Q  Extremely  poor  Q Poor  Q  Satisfactory  CB'Good  Q  Excellent 


3.  Rate  your  ability  to  understand  the  synthetic  voice  messages. 

O  Extremely  poor  CD  Poor  Qifatisf  actory  0  Good  O  Excellent 

4.  Rate  the  quality  of  radio  voice  in  the  cockpit  during  actual  flight. 

0  Extremely  poor  QlPoor  O  Satisfactory  O  Good  Q  Excellent 


5.  Rate  the  quality  of  radio  voice  in  this  study. 

O  Extremely  poor  »0Poor  (^Satisfactory  0  Good  O  Excellent 


6.  Rate  your  ability  to  perceive  your  radio  call  sign. 

O  Extremely  poor  Qpoor  Q'Satisf  actory  O  Good  o  Excellent 


7.  Rate  your  ability  to  understand  the  radio  message  following  your  call  sign. 
O Extremely  poor  0Poor  O  Satisfactory  0  Good  0  Excellent 
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TEST  SUBJECT  RATING  SHEET  (Cont'd.) 


8. 


Did  you  encounter  any  problems  in  attending  to  and/or  understanding 
simultaneous  messages  (synthetic  and  radio  voice)? 


G  Yes 

C9"No 


If  your  answer  is  Yes.  please  elaborate 


Use  the  space  below  for  providing  additional  comments  and/or 
evaluations. 
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